Skip to content

Implement context accessor for DatasetEvent extra#38481

Merged
uranusjr merged 5 commits into
apache:mainfrom
astronomer:dataset-event-extra-context
Mar 29, 2024
Merged

Implement context accessor for DatasetEvent extra#38481
uranusjr merged 5 commits into
apache:mainfrom
astronomer:dataset-event-extra-context

Conversation

@uranusjr

@uranusjr uranusjr commented Mar 26, 2024

Copy link
Copy Markdown
Member

See #37810 for details. This implements the dataset_events context key to add extra to a given dataset event that’s to be emitted by the task.

I can use some help with how to do the UI part.

Comment thread airflow/datasets/__init__.py Outdated
@uranusjr uranusjr force-pushed the dataset-event-extra-context branch from 54834d2 to 8380a93 Compare March 26, 2024 07:36

@jscheffl jscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the approach as proposed here. Besides the comments to the review, can I kindly request that the existing dataset example is extended such that the "new function" is used there as a good example? Benefit would also be that there is an example existing how to retrieve the extra when receiving the event. At the moment it is a bit complex for somebody trying to figure this out.

Comment thread airflow/models/taskinstance.py Outdated
Comment thread docs/apache-airflow/authoring-and-scheduling/datasets.rst Outdated
Comment thread docs/apache-airflow/authoring-and-scheduling/datasets.rst Outdated
Comment thread scripts/ci/pre_commit/pre_commit_template_context_key_sync.py Outdated
@uranusjr uranusjr force-pushed the dataset-event-extra-context branch from 47da435 to 2996db3 Compare March 28, 2024 23:42
@uranusjr uranusjr merged commit fce3a58 into apache:main Mar 29, 2024
@uranusjr uranusjr deleted the dataset-event-extra-context branch March 29, 2024 02:06
@ephraimbuddy ephraimbuddy added the type:new-feature Changelog: New Features label Mar 30, 2024
@vatsrahul1001 vatsrahul1001 added this to the Airflow 2.10.0 milestone Jul 12, 2024
@bbovenzi

Copy link
Copy Markdown
Contributor

Covered in: #40871

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants